Publish the three hub pages as Markdown twins - #168
Conversation
/resources.md, /compare.md, and /alternatives.md answered 404 while every page below them answered 200. An agent that reached /compare/claude-skills-vs-plugins.md could read that page and nothing above it: the descent from a hub to a page existed only in HTML, and there was no climb back at all. lib/seo/hubs.ts adds the three hubs as content definitions, built from the same registries the HTML hubs render, so a page added to a registry appears in the hub twin with no change there. Each twin states what the hub indexes and then lists every page under it with the summary the hub card already shows, and the three link to each other. The twins follow the conventions the comparison pages got in #165: the registry in lib/markdown/twins.ts, an entry in the WebMCP page catalogue, the text/markdown alternate on the HTML page, and a content negotiation rewrite so the page URL answers in Markdown when the request asks for it. The generic <path>.md rewrite already routed the three URLs, so only the Accept rules are new. The twins stay out of the sitemap, which lists the HTML page only, and llms.txt gains the alternatives hub and names the three hubs in the sentence that says which pages have a Markdown version.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (10)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. WalkthroughThe change adds Markdown twins for the resources, alternatives, and comparison hubs. It derives hub content from shared registries, adds content-negotiation rewrites and alternate metadata, updates catalogues and ChangesMarkdown hub twins
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: ⚪ Minimal · up to This change adds Markdown representations for three existing public hub pages and keeps their HTML, sitemap, and related page behavior intact. No actionable merge-blocking risk remains beyond normal checks and review. Sequence Diagram(s)sequenceDiagram
participant Client
participant NextConfig
participant MarkdownAPI
participant TwinRegistry
Client->>NextConfig: Request hub with text/markdown
NextConfig->>MarkdownAPI: Rewrite path to /api/markdown
MarkdownAPI->>TwinRegistry: Resolve hub twin
TwinRegistry-->>MarkdownAPI: Return Markdown document
MarkdownAPI-->>Client: Return Markdown response
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Description checkExplanation The description explains the problem, implementation, expected impact, success metric, and detailed validation results. It uses headings that differ from the template and does not provide a separate Impact section, but the required information is mostly present and the description is complete. Full details: Docstring CoverageExplanation Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 9 files. (1 skipped: 1 unsupported.) ✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 15786e68d4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| modifiedAt: lastModified(resourceEntries), | ||
| intro: [ | ||
| "This page is the index of the Skills Board resource pages: guides, skill format explainers, and reference pages for teams that share and operate AI skills. It answers nothing itself, and each entry below is the page that answers its own question.", | ||
| "Every page listed here names its publisher, its first-party sources, and the date its claims were last checked. Every one of them is also published as Markdown at the same URL with a `.md` suffix.", |
There was a problem hiding this comment.
.md promise to pages that have twins
When an agent treats this sentence as routing guidance, two pages listed later in this document violate it: Related resources includes /about and /llms.txt, but neither is registered in twinEntries, so /about.md and /llms.txt.md fall through the generic rewrite to a 404. This makes the new hub advertise dead Markdown URLs; either restrict the promise to the indexed topic entries or publish/link actual twins for every listed page.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Founded. The sentence covered the whole document while the Related resources block links /about and /llms.txt, and neither has a twin: /about.md answers 404 in production, checked before the fix. Commit c2b3365 scopes the promise to the topic entries, which all carry twins, and adds a contract test that fails whenever a hub lists a page without a Markdown twin, for all three hubs. The /about and /llms.txt links stay in Related resources as plain links, now outside the promise. Typecheck 0, 562 tests green.
|
@CodeRabbit review |
✅ Action performedReview finished.
|
The intro sentence claimed every page listed on the hub is also published as Markdown, but the Related resources block lists /about and /llms.txt, and neither has a twin: /about.md answers 404 in production. The promise now covers the topic entries, which all carry twins, and a new contract test fails if a page ever appears in a hub list without one. The /about link note also drops the pre repositioning wording.
1 participant
What this changes
/resources.md,/compare.md, and/alternatives.mdanswered 404 in production while every page below them answered 200. This branch publishes the three hub pages as Markdown twins, following the conventions the comparison pages got in #165.lib/seo/hubs.ts: the three hubs as content definitions, built from the same registries the HTML hubs render, so a page added to a registry appears in the hub twin with no change there. Each twin states what the hub indexes, then lists every page under it with the summary the hub card already shows, and the three hubs link to each other.lib/markdown/twins.tsandlib/web-mcp-pages.ts: each hub is registered immediately above the collection it lists, so the twin registry and the WebMCP page catalogue read in the order an agent walks them.next.config.ts: one content negotiation rule per hub, so/resources,/compare, and/alternativesanswer in Markdown when the request sendsAccept: text/markdown. The generic<path>.mdrewrite already routed the three.mdURLs, so only theAcceptrules are new.<link rel="alternate" type="text/markdown">throughmarkdownTwinAlternates, the way every other page with a twin does.public/llms.txtgains the alternatives hub and names the three hubs in the sentence that says which pages have a Markdown version..mdURL is in it.Hypothesis
The agent-facing graph had no hub nodes. An agent that arrived at
/compare/claude-skills-vs-plugins.md, from llms.txt, from a citation, or by appending.mdto a URL it was given, could read that page and nothing above it: the hub that lists the other three pairs answered 404 in Markdown. The descent from a hub to a page existed only in HTML, and the climb back did not exist at all.Adding the hub twins closes the loop in both directions: from a hub twin an agent reaches every child, and from a child it can now climb to the hub, then across to the two sibling hubs. The claim being tested is that a complete Markdown graph gets more of the cluster read per visit than a set of disconnected leaf documents does.
Metric this should move
Server-side fetches of the three hub twins, and agent-facing coverage: the share of public content pages that answer in Markdown goes from every leaf to every leaf plus the three nodes above them. Secondary signal: fetches of child twins that arrive after a hub twin fetch from the same client, which is the descent the hub is there to enable.
No direct SEO metric. The twins are not in the sitemap, are not separate indexable documents, and the canonical URL in every twin header is the HTML page. Nothing here is expected to move impressions or clicks in Search Console.
How we will know it worked
At two to four weeks: 200 responses on
/resources.md,/compare.md, and/alternatives.mdin the server logs, from clients other than our own checks. Zero is the current baseline, since the three URLs answered 404 until this branch. If the hub twins are fetched but no child twin fetch follows from the same client, the hub is being read and the graph is still not being walked, which is a content problem in the hub twins rather than a routing one.Verification
tsc --noEmit: 0 errors.node --test tests/*.test.mjs: 561 pass, 0 fail. 10 of them are the new contract tests intests/hub-markdown-twins.test.mjs: each hub has a twin at its own path plus.md, the twin links to every page the hub indexes and says what each one is about, the three hubs reach each other, the.mdURL and the negotiated page URL return the same document through the route handler with and without the rewrite query, the negotiation rule exists, the HTML page advertises the alternate, WebMCP lists each hub above its own collection, llms.txt lists the three, and no.mdURL is in the sitemap.next build: exit 0.next start:/resources.md200 (10628 bytes, 30 absolute links),/compare.md200 (2352 bytes, 7 links),/alternatives.md200 (1904 bytes, 7 links), alltext/markdown; charset=utf-8. The same three paths without.mdand withAccept: text/markdownanswer 200text/markdown, and answer HTML without the header. Non-regression:/compare/claude-skills-vs-plugins.md,/index.md,/codex-skills.md,/pricing.md,/llms.txt, and/sitemap.xmlall still 200,/nope.mdstill 404, and the sitemap contains no.mdURL.Summary by CodeRabbit
New Features
Bug Fixes
Tests